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Abstract 

Basic properties of Brownian motion are used to derive two results concerning 
birth-death chains. First, the probability of extinction is calculated. Second, 
sufficient conditions on the transition probabilities of a birth-death chain are 
given to ensure that the expected value of the chain converges to a limit. 
The theory of Brownian motion local time figures prominently in the proof 
of the second result. 
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1. Introduction 

Let X rn be a Markov chain taking values on the nonnegative integers with 
the following transition probabilities for n ^ 



Implicit here is the fact that r n + l n — 1. We suppose further for simplicity 
that Xq = k almost surely, for some k G N. X m is essentially a random walk 
on the nonnegative integers, moving to the right from state n with probability 
r n and to the left with probability l n . We refer to such a Markov chain as a 
birth-death chain. This name comes from considering X m as the number of 



(1) 




if j — n + 1 
if j — n — 1 
if \ n -j\^l. 
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members in a population, where at each step either a new member is born or 
an old member dies, causing the process to increase or decrease by 1. We can 
assume poo = 1 and poj = for any j ^ 0, as when the population reaches 
it is considered to have gone extinct with no possibility of regeneration. The 
purpose of this paper is to introduce a method of using properties of Brownian 
motion to deduce two fundamental theorems concerning birth-death chains. 
The first theorem, presented in the next section, gives the probability that 
a birth-death chain goes extinct at some finite time. The second theorem, 
presented in Section 3, gives sufficient conditions for E[X m ] to converge as 
m — > oo. The properties of Brownian motion which will be utilized are 
standard and can be found in many references on Brownian motion, such as 

or pq. 

We will now introduce the basic setup. Let to := 1 and 
(2) t n := 

rir 2 . . . r n 

for n > 0. Define a sequence {x„}^ =0 recursively by setting x = 0, and 
having defined x n let x n +i = x n + t n . Since the sequence {x n } is increasing 
it converges to a limit Xqo, possibly infinite, as n — > oo. Let B t be a 
Brownian motion starting at Xk and stopped at the first time Ta it hits 
or x^. The recurrence properties of Brownian motion imply that < oo 
almost surely. We define a sequence of stopping times T m which are, roughly 
speaking, the successive hitting times of A := {x n }™ =0 . More rigorously, T m is 
defined recursively by setting T = 0, and having defined T m we let T m+i = 
mf t> T m {B t G A, B t 7^ Br m }. We see that the variables Bt , B^, Bt 2 , ■ ■ ■ 
form a random process taking values in A. The strong Markov property of 
Brownian motion, together with the standard exit distribution of Brownian 
motion from an interval, imply that 



P ( B T m+1 = x n+ i\B Tm =x % 
and, likewise, 



2<n ^n— 1 ^n— 1 1 



•^71+1 ■En—1 t n —i ~\- t n 1 -|- l-n/^r. 



P(B Tm+1 = X n -l\B Tm = X n ) = l n 
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If we define 0on {x n }^L by <f>(x n ) = n, we see that (f>(B To ), <p(B Tl ), (p(B T2 ), . . . 
is a realization of our original birth-death chain. The picture below gives an 
example, where we have oriented the time axis vertically and the space axis 
horizontally. 



X k-2 X k-1 X k X k+1 X k+2 

Figure 1: The Brownian path pictured realizes the birth-death path k, k + 1, k, k + 
1 j /c , Jv 1 5 k ; k 1 , k 2 , . . . 

Given this framework, we are ready to prove several theorems. In the sequel, 
any reference to X, B, x n , Ta, 0, etc. will refer to the definitions presented in 
this section. 

2. The extinction probability of a birth-death chain 

Perhaps the most fundamental question one can ask regarding a birth-death 
chain is whether the population must go extinct or not, that is, whether 

P(X rn = for some m) — 1 or P(lim m >OQ X m = +oo) > 0. Let P k be the 

probability that the birth-death chain eventually hits (recall X = k a.s.). 
We then have the following. 



Theorem 1. 

(3) 



V°° t ■ 
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where this quotient is interpreted as being equal to 1 if the sums diverge. 

This elegant theorem has a straightforward proof using recurrence relations; 
see [3] or [5]. A potentially pleasing aspect of the proof below, however, lies 
in giving a clear, visual intuition for the sums in ([3]). 

Proof of Theorem [TJ Recall that Xoo = lim n yoo x n is given by 

oo 

(4) Xoq = ^ ] tj 

j=0 

If = oo, so that both sums in (J3]) diverge, then B Ta = almost surely. 
This implies that the population dies out with probability 1. On the other 
hand, if Xoo < oo then P(Bt a = 0) is given by 

Eoo , 

^ ' ~x ^o" = V°° 

However, as in the first case, P(Bt a = 0) is precisely Pk, the probability 
of extinction. This is because the Brownian motion hitting x,^ before 
implies B Tm — > x^, hence (p(B Tm ) — > oo, whereas hitting before x^ 
implies 4>{B Ta ) = for some m. The two cases (x^ = oo and x^ < oo) are 
illustrated in the following figure. 
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Xj x 2 x 3 x 4 x 5 x 6 Xj x 2 x 3 x 4 x 5 x 6 ... x c 



Figure 2: The left panel illustrates the situation in which Yl'jLo *j diverges. Even- 
tually, B t hits and the population goes extinct. The right panel illustrates the 
other scenario, in which YlJLo^j = x °° < 00 ■ ^ n this case, there is a positive 
probability that B t hits Xqo before 0, in which case the population never goes 
extinct. 

This completes the proof of Theorem 1. □ 

3. The long-term average of a birth-death chain 

Recall that B and X are stopped upon reaching 0. It will therefore be 
convenient to let X m be denned to be for all m > m , where mo is the 
smallest integer, if it exists, for which X mo = 0. Similarly, for convenience 
let T m = Ta for all m > m , where m is the smallest integer, if it exists, 
for which -&r mo = 0. In the case T{ — U — \ for all i, it is well known that 
X m is a martingale, and therefore £[X m ] = Eq = k for all m. This occurs 
despite the fact that P(X m = 0) — > 1 as m — > oo, as the average value 
of X m on {X m 7^ 0} grows at exactly the right speed to balance the set of 
large probability upon which X m = 0. Such behavior certainly does not hold 
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for the general case, since we no longer have the martingale property, but we 
will see that the Brownian motion model presented above can shed light on 
the behavior of _E[X m ] as m — > oo. 

Recall that A = {x n }^ =0 . Let : A — > 1R + be extended to a continuous 
function from IR + to 1R + by defining to be linear on each interval x n ). 
Alternatively, we may think of x n = x(n) as a function from N to 1 which 
can be extended by linear interpolation to an increasing function from IR + to 
]R + . In this case, is simply x~ x . is therefore a piecewise linear function, 
and 0' exists on IR + — A. Let 0' n be the value of 0' on (x n -i,x n ). We will 
prove the following theorem. 

Theorem 2. If 0'^ = lim n >oc 0' n exists, then 



(6) lim E[X m ]=x k <j>' 00 

m — >co 

Note that we are allowing 0^ = +oo or 0. Writing 0'^ and x k in terms of the 
Z n 's and r n 's shows that the following statement is equivalent to Theorem [2j 

If t^ '■= h m n >oo ~~ exists then lim m >OQ E[X m ] exists, and 

(7) Km £[X m ] = n 



The bulk of the rest of this section is devoted to the proof of this theorem. 
We will simplify initially by assuming Y^m=i Wn+i ~ 4>'n\ < 00 5 ^is condition 
will be removed at the end of the proof. For the case in which there is a 
positive probability that the population never goes extinct, it is easy to see 
that E[X m ] — > oo as m — > oo, and that 0'^ exists and is equal to +oo, so 
that ^ is valid. We will therefore assume that P(X m = for some m) = 1. 
Note that 0^ +1 = i and x n+ \ — x n = t n , so that 4>' n+1 (x n+ i — x n ) = 1. Note 
also that x\<$ 1 = 1. This allows us to perform the following manipulations to 
obtain an expression which will be more convenient for the purposes of the 
proof. 



k-l 

(8) Xfc0' oo = Xfc(0' oo - (j)' h ) + X k (f)' k + k — XKj)[ - <f>n+l(Xn+l ~ X n ) 

n=l 
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fc-1 



n=l 



The last equality uses summation by parts; see pp. We see that the conclusion 
of the theorem is equivalent to showing 



fc-i 



(9) lira E[X m ] = k + x^ - fa) + ^(0'„, +1 - <j>' n )x n 

n=l 

This is what we will prove. We will proceed through several lemmas, and will 
need properties of Brownian motion local time, which is the density of the 
occupation measure of Brownian motion with respect to Lebesgue measure. 
That is, the local time L| satisfies 

(10) L x t dx = [ l Bs £dxds 



It is well known that L x exists and that 



1 « 



(11) L x t = e lim— y l\B B -x\< e ds 

almost surely. See [2] for a comprehensive treatment of local time, or the 
more general reference [3] . The following is an extension of Tanaka's formula, 
Theorem VI. 1.2 in jl]. 

Lemma 1. Almost surely, for any stopping time T , 

(12) <f>(B T ) = k+ T 4>'(B s )dB s + 

^° n=l 

where L^ n denotes the local time of B t at x n at time T . 

Proof: Note that <f>"(x) = 5^^=i(0n+i — < P'n)^3> n ( x ) i n the sense of distribu- 
tions, where S Xn (x) = 5o(x — x n ) denotes the Dirac delta function at point 
x n . Lemma [l] is therefore seen to be a special case of the Ito-Tanaka formula, 
Theorem VI. 1.5 of [4J, provided that <fi can be realized as a difference of two 
convex functions. However, any piecewise-linear function can be realized as 
the difference of two convex functions, provided that the points of nondiffer- 
entiability do not accumulate. We may argue as follows. 0' is a piecewise 
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constant function, which is therefore of bounded variation on bounded inter- 
vals, and as such we may write (ft — f — g where / and g are nondecreasing. 
Let F and G be antiderivatives of / and g chosen so that <fi = F — G. Then 
F and G are convex, and the result follows. □ 

Applying this lemma to the stopping time T m , we immediately obtain 
(13) E[X m ) = k + jr ^ E[L*l], 

71=1 

Note that the convergence of the sum at this point is not an issue, since L?f = 



for n > m + k. Using the identity (13) does not seem to be an effective 



way to calculate £ , [X m ], due to the difficulty of obtaining information about 
T m . Nonetheless, we do know that T m /* Ta as m — > oo, and this implies 



u 



(14) hm E[X m \ =k + J2 " n+ \ Vn> E[L%] 



n=l 



provided that E[L^] can be bounded uniformly, which we will show soon to 



be the case. We should mention that it was in obtaining (14) that we used 
the assumption that Y^=i Wn+i ~ 0nl < 00 • This is because a priori the 
quantities E[L^ ] may be growing in some strange way that causes problems 
^ Sn=i I — 4>'n\ = 00 • We will return to this point at the end of the proof. 



In light of (14), we must compute E[L^]. 



Lemma 2. 

(15) E[L%] = 2 min(x k ,x n ) 

Proof: One may derive this through standard calculations involving the 
probability density function of B t , but the following is a quicker and easier 
proof. Let us suppose first that n = k. From Tanaka's formula, E[L^] = 

lim t >OQ E[\B t — Xk\]. Furthermore, B t is a martingale, so E[(B t — x^)] = 

for all t. It follows from this that 



(16) E[L%] = 2 lim E[max(-(B t - x k ), 0)] 

t — >oo 



2 lim (x k P{B t = 0) + / (x k - x)P{B t e dxj) 
*— v Jo ' 
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As limj >00 P(B t = 0) = 1, we can conclude that E[L^} = 2x k . Now suppose 

n 7^ k. Note that, if we let T Xn = inf{t : B t = x n }, then 



(17) Z4« = L Txn + L^B o e T Jl Txn<TA = L^B o 9 T Jl Txn<TA 

where 9 denotes the standard shift operator and L t {B o 9r Xn ) is the local 
time of the shifted process B o Q Txn . Let E Xj denote expectation with respect 
to a Brownian motion W which starts at Xj and is stopped upon hitting 0. 



The prior calculation together with (17) and the strong Markov property of 
Brownian motion imply that 



(18) E[I%] = P{T Xn < T A )E Xn [L%] 

= P(T Xn < T A )2x n 

The general result follows from noting that P(T Xn < Ta) is 1 if x n < Xk and 
if x n > x k . □ 



Combining (14) and Lemma |2| gives 

fc— 1 oo 
(19) lim E[X m ] = k + Ys^'n+l ~ #>n + X k ^(0' n+1 - <#J 

n=l n=k 

Since Yl^Lkiftn+i ~~ 0n) = (0oo ~ 4>'k)i we are done in this case. It remains 
only to remove the restriction that Y^n°=i Wn+i ~ $J < 00 • The following 
lemma is key. 

Lemma 3. For any m, and any n> k, E[L^} > E[L^ +1 ]. 

Proof: In fact, we may prove somewhat more, namely that if T is any 
stopping time with B T e A almost surely, then E[L^ n ) > E[L^ n+1 ]. Let 
E x . and W be as in the proof of Lemma |2j Using Lemma [2] and the strong 
Markov property of Brownian motion, we obtain 



(20) E[L^]=E[L%}-E[lim Q -j T l ( _ e , e) (B, - x n )ds 



2x k -^P{B T = x j )E x 



n-l 



1 f°° 

lim — / li ££) (W s -x n )ds 



2x k - P(Bt = Xj ) Xj - P (Bt = x 3 ) 



x , 



j=n 
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Similarly, 



(21) E[L. 



T J 



2x k ~ ^2 P ( Bt = x ^ x i ~ E P ( Bt = 

j=l j=n+l 

The conclusion of the lemma now follows from the fact that x n+ \ > x n . □ 



We may now complete the proof of the theorem. Recall (13), and observe 
that E[L^] = for n > k + m, since X m < m + k. This means that (13) is 
in fact a finite sum. 



m+k 



(22) E[X m ] = k+J2 



n=l 
fc-1 



n=l 



e[l x t : 



k+m i ,f 



-E[L% 



n=k 



The indices of the first sum in the final expression of ( 23 ) are independent of 
m. This implies that the sum converges as m — > oo, since E[L^ n ] — > 2x n 
as m — > oo for n < k. We must show that the second sum converges as 
m — > oo. We use summation by parts again, which gives 



k+m /,/ I, v 

(23) £ 2 E ^ L tJ = 2 K m+ i^rr +1 ] " 

n=k 

k+m 

-E^ + i(^[^: +1 ]-^[^:])) 

n=k 

Let us assume that cf)'^ < oo, and let e > be given. We may choose 
N > k such that (f>' n e (cj)^ — e, 4>oo + e) for all n > N. Having chosen 
this, we may choose M > N — k such that 2x k > E\Lj?\ > 2xk — e for 
all n G [k,N),m > M. Using the fact that E[L^ k + m+1 ] = 0, and setting 
<p' = sup J>0 <p'j, we see that for m > M 



k+m _ AJ \ 

(24) J2 e [ l tJ 

n=k+l 
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n=N+l 

H^ + e)(E[LZ +1 ]-E[L^]) 
< X - ( - ^(2** + (0'oe + e)2x k 



This shows that 

fc+m (Aj _ fA' ^ 

(25) limsup £ 1 , — Sfr%J < - 0',) 

m — >oo , * 
n=fc 

Proceeding similarly, we can obtain 

( 26 ) irisf S 2 — > - 

n=fc+l 

Together, these prove the desired convergence. The case 4>'oo = +oo is similar 
but easier and is omitted. This completes the proof of Theorem [2} □ 

We conclude with a simple but counterintuitive example. Let l n = 2^b[> r n = 
for n > 1. Then t n = ^-j-, so that = 0. On the other hand, = 
1 + Y^=\tn = oo. We see that the birth-death chain X m built upon these 
transition probabilities has an extinction probability of 1, but £[X m ] — > oo 
as m — > oo. 



4. Acknowledgements 

I am grateful for support from the Priority Research Centers Program through 
the National Research Foundation of Korea (NRF) funded by the Ministry 
of Education, Science and Technology (Grant #2009-0094070). I would also 
like to thank the referee for comments which improved the exposition, as well 
as for asking an interesting question which led to the development of Section 
3. 



11 



5. References 
References 

[1] Lang, S., 1997, Undergraduate Analysis, Springer- Verlag, New York. 

[2] Marcus, M., Rosen, J., 2006, Markov Processes, Gaussian Processes, and 
Local Times, Cambridge University Press, New York 

[3] Norris, J. R., 1997, Markov chains, Cambridge University Press, New 
York. 

[4] Revuz, D., Yor, M., 1968, Continuous Martingales and Brownian Motion, 
third edition, Springer, Berlin. 

[5] Syski, R., 1992, Passage Times for Markov Chains, IOC Press, Amster- 
dam. 



12 



